Methods and apparatus for estimating unknown quantities

ABSTRACT

A method and apparatus for estimating unknown quantities provide plural sets of hierarchical perceptrons 1i in parallel to one another, construct a condition by learning in which a processing equivalent to a numerical formula corresponding to relationship between input and output in each hierarchical perceptron 1i, then construct a partial differential function of the numerical formula by learning based upon a difference between a cumulative addition value of function value outputs and a teacher pattern, and a difference between a partial differential value corresponding to the function value output and a partial differential value teacher pattern. Thereafter, the method and apparatus cumulatively add outputs from the plural hierarchical perceptron 1i, then correct input layer corresponding to unknown quantities of the hierarchical perceptron 1i based upon a difference between the cumulative addition value and the measurement value and a partial differential value of the cumulative addition value. The method and apparatus collect corrected values in the input layer and output the corrected values as unknown quantities estimated values at a timing that the difference becomes sufficiently small.

TECHNICAL FIELD

This invention relates to methods and apparatus for estimating unknownquantities. More particularly, the present invention relates to methodsand apparatus for estimating unknown quantities under a rule whichcannot be clearly expressed by numerical formulae or a rule which can beexpressed by numerical formulae, and which requires much labor foractually calculating the numerical formulae.

BACKGROUND ART

From past years, a so called artificial neural network is known in whichplural neuron devices are classified into multiple layers, each neurondevice belonging to each layer is connected to all neuron devicesbelonging to other layers, and by supplying an input pattern to neurondevices belonging to an input layer, a predetermined output pattern isoutput from neuron devices belonging to an output layer, thepredetermined output pattern being determined based upon the coefficientof coupling and threshold value of each neuron device. The artificialneural network is called as a hierarchical perceptron because the neurondevices constitute a hierarchical arrangement in the artificial neuralnetwork.

In the hierarchical perceptron, a so called learning is repeated. Thelearning is performed by supplying a known input pattern to a group ofneuron devices of the input layer and supplying a known output patterncorresponding to the known input pattern as a teacher pattern, bycalculating a difference between the teacher pattern and an outputpattern output from the neuron devices belonging to the output layerbased upon the input pattern, and by varying the coefficient of couplingand the threshold value of each neuron device so as to decrease thecalculated difference. After carrying out sufficient learning, when anarbitrary input pattern is supplied to the neuron devices belonging tothe input layer, an output pattern which should correspond to the inputpattern is output from the neuron devices belonging to the output layer.That is, when at least a part of a plurality of input data of an inputpattern is varied, corresponding unknown quantities are obtained as anoutput pattern.

The hierarchical perceptron is determined from its kinds and number ofthe data of input pattern, and when the output pattern is known whichshould be output corresponding to a known input pattern, after carryingout the learning by a necessary number of times using these known data,an output pattern corresponding to an arbitrary input pattern isobtained. Therefore, since the learning should be carried out again whenthe kind and number of data of the input pattern are varied, adisadvantage arises that varying in applicable extent cannot be easilydealt with.

Further, when the number of data of the input pattern increases, and thenumber of neuron devices belonging to each layer or the number of layersis not increased, accuracy of the output pattern is lowered, thereforethe number of neuron devices constituting the hierarchical perceptron isgreat, and a disadvantage arises that a required time for learning islengthened caused by the number of the neuron devices being great.

Furthermore, when a part of the input pattern and data which should be atrue output pattern are obtained by a conventionally known measurementmethod and the like, a disadvantage arises that obtaining the rest ofthe input pattern is impossible caused by the arrangement of thehierarchical perceptron.

The present invention was made to solve the above-mentioned problems. Itis an object of the present invention to supply novel methods andapparatus for estimating unknown quantities, the methods and apparatusenabling estimating unknown quantities under a rule which cannot beclearly expressed by numerical formulae or a rule which can be expressedby numerical formulae, and which requires much labor for actuallycalculating the numerical formulae.

DISCLOSURE OF THE INVENTION

To perform the object above-mentioned, a method for estimating unknownquantities according to a first embodiment of the invention comprisesthe steps of:

supplying arbitrary quantities corresponding to unknown inputs andcommon known inputs as input values and a predetermined measurementvalue as a teacher pattern to plural artificial neural networks each ofwhich has a multiple layers arrangement and has finished learning of apredetermined function,

correcting the arbitrary quantities corresponding to the unknown inputsin each artificial neural network so as to decrease a difference betweena cumulative addition value and the teacher pattern based upon the inputvalues, the difference between the teacher pattern and the cumulativeaddition value of outputs from the plural artificial neural networks,and a partial differential values corresponding to the correspondingartificial neural network, and

obtaining quantities which are corrected so as to reduce the differenceto the least value as estimated results of unknown inputs.

As to the method for estimating unknown quantities according to claim 1,plural sets of artificial neural networks are provided, each artificialneural network having performed learning so as to obtain a predeterminedoutput corresponding to rather simple multiple data of an input pattern.Arbitrary quantities corresponding to unknown inputs and known inputsare supplied as input values and a predetermined measurement value issupplied as a teacher pattern to the plural sets of artificial neuralnetworks, and correction of the input layer of each artificial neuralnetwork is carried out so as to decrease the difference between thecumulative addition value and the measurement value. And, correctionresults of the arbitrary quantities corresponding to the unknown inputsof each artificial neural network are obtained as unknown quantitiesestimation results when the difference becomes sufficiently small.

Therefore, the scale of each artificial neural network should not beenlarged too great so that a required time is extremely shortened whichis required for learning of a function at a first stage for eachartificial neural network.

A method for estimating unknown quantities according to anotherembodiment of the invention comprises the steps of:

obtaining plural function value output by supplying known inputs toplural artificial neural networks each of which has a multiple layersarrangement and has finished learning of a predetermined function,

obtaining a difference between a cumulative addition value of eachobtained function value output and a measurement value previouslyobtained in correspondence to the known inputs, and a difference betweenpartial differential values corresponding to each function value outputand partial differential values previously obtained in correspondence tothe known inputs,

learning a partial differential function in each artificial neuralnetwork based upon the calculated difference between the partialdifferential values, the known inputs, and the difference between thepredetermined measurement value previously obtained in correspondence tothe known inputs and the cumulative addition value,

repetitively carrying out the learning of the partial differentialfunction until the difference between the predetermined measurementvalue and the cumulative addition value and the difference between thepartial differential values become sufficiently small,

supplying arbitrary quantities corresponding to unknown inputs andcommon known inputs as input values and a predetermined measurementvalue as a teacher pattern to all artificial neural networks,

correcting the arbitrary quantities corresponding to the unknown inputsin each artificial neural network so as to decrease a difference betweena cumulative addition value and the teacher pattern based upon the inputvalues, the difference between the teacher pattern and the cumulativeaddition value of outputs from the plural artificial neural networks,and a partial differential value corresponding to the correspondingartificial neural network, and

obtaining quantities which are corrected so as to reduce the differenceto the least value as estimated results of unknown inputs.

As to the method for estimating unknown quantities according to thisembodiment, plural sets of artificial neural networks are provided, eachartificial neural network having performed learning so as to obtain apredetermined output corresponding to rather simple multiple data of aninput pattern. Known quantities are supplied to each artificial neuralnetwork instead of the unknown quantities which are objected forestimation. The difference between the measurement value previouslyobtained in correspondence to the known quantities and the cumulativeaddition value of the function value outputs from each artificial neuralnetwork, and the difference between the partial differential values eachof which corresponds to the function value output from each artificialneural network and the partial differential values (partial differentialvalue teacher pattern) each of which is previously obtained by numericaldifferential method and the like are obtained. By carrying out learningof the partial differential function in each artificial neural networkbased upon the difference between the teacher pattern previouslyobtained in correspondence to the known inputs and the cumulativeaddition value, and the difference between the teacher pattern and thepartial differential values, pre-processing of an unknown quantitiesestimating system in its entirety which includes the plural sets ofartificial neural networks, is finished. Thereafter, arbitraryquantities corresponding to unknown inputs and known inputs, and ameasurement value are supplied to the plural sets of artificial neuralnetworks, and correction of the input layer of each artificial neuralnetwork is carried out so as to decrease the difference between thecumulative addition value and the measurement value. And, correctionresults of the arbitrary quantities corresponding to the unknown inputsof each artificial neural network are obtained as unknown quantitiesestimation results when the difference becomes sufficiently small.

Therefore, the scale of each artificial neural network should not beenlarged too great so that a required time is extremely shortened whichis required for learning of a function at a first stage for eachartificial neural network. And, a required time for learning the partialdifferential function is also shortened because learning of the partialdifferential function in each artificial neural network which hasfinished learning of the function in such manner is carried out basedupon the difference between the partial differential value correspondingto the function value output and the partial differential value (partialdifferential value teacher pattern) which was previously obtained, andthe difference between the measurement value which is known informationand the cumulative addition value of the estimated values of eachartificial neural network.

The known inputs, for example measurement condition and the like, andarbitrary quantities corresponding to unknown inputs are supplied asinput values and the measurement value is supplied as a teacher patternto the plural sets of artificial neural networks which have finished thepre-processings (the learning of the function and the learning of thepartial differential function), and in this condition, correction of aninput layer for inputting unknown quantities of each artificial neuralnetwork is carried out so as to decrease the difference between thecumulative addition value of the function value outputs from allartificial neural networks and the teacher pattern. Therefore,estimating of unknown quantities which was not possible at all in pastyears, is easily performed.

An apparatus for estimating unknown quantities according to anembodiment of the invention comprises:

plural artificial neural networks each of which has a multiple layersarrangement and has finished learning of a predetermined function,

partial differential value calculating means for calculating a partialdifferential value corresponding to a function value output of eachartificial neural network,

cumulative addition means for cumulatively adding the function valueoutputs each of which is output from each artificial neural network,

first difference calculating means for calculating a difference betweena predetermined teacher pattern which was previously obtained and thecumulative addition value,

second difference calculating means for calculating a difference betweena partial differential value corresponding to the function value outputand a partial differential value which was previously obtained, forevery artificial neural network,

partial differential function learning means for carrying out learningof a partial differential function based upon the difference between thepartial differential values, calculated by the second differencecalculating means, and the difference between the teacher pattern andthe cumulative addition value, calculated by the first differencecalculating means,

input unknown quantities correcting means for correcting arbitraryquantities corresponding to unknown inputs and for obtaining thecorrected quantities which reduce the differences to the least values,as estimated results of unknown quantities, in response to allartificial neural networks having been performed learning of the partialdifferential function by the partial differential function learningmeans so as to lessen the difference calculated by the first differencecalculating means and the difference calculated by the second differencecalculating means below a predetermined value, wherein all artificialneural networks are supplied arbitrary quantities corresponding to theunknown inputs and the common known inputs as input values so as toobtain the function value outputs and the cumulative addition value ofthe function value outputs, such that each artificial neural networkcorrects the arbitrary quantities so as to decrease the differencebetween the cumulative addition value and a teacher pattern based uponthe difference between the cumulative addition value and the teacherpattern, and a partial differential value corresponding to the functionvalue output of the corresponding artificial neural network, and

information taking out means for taking out the quantities which arecorrected by the unknown quantities correcting means.

As to the apparatus for estimating unknown quantities according to thisembodiment, by supplying the known inputs and the known teacher patternto each artificial neural network which has a multiple layersarrangement and has finished the learning of the predetermined function,the difference between the cumulative addition value of the functionvalue outputs and the teacher pattern is obtained by the firstdifference calculating means, and the difference between the partialdifferential value which is calculated by the partial differential valuecalculating means and the partial differential value (partialdifferential value teacher pattern) which is previously obtained bynumerical differential method and the like is obtained by the seconddifference calculating means. Learning of the partial differentialfunction is carried out in each artificial neural network by the partialdifferential function learning means based upon the difference betweenthe obtained cumulative addition value and the teacher pattern, and thedifference between the partial differential values. And, after thelearning of the partial differential function is carried out by thepartial differential function learning means until the differencebetween the cumulative addition value and the teacher pattern and thedifference between the partial differential values become sufficientlysmall, arbitrary quantities corresponding to the unknown inputs arecorrected by the unknown quantities correcting means so as to decreasethe difference between the cumulative addition value and the teacherpattern based upon the partial differential value corresponding to thefunction value output of the corresponding artificial neural network andthe difference between the teacher pattern and the cumulative additionvalue of the function value outputs which are obtained by supplying thearbitrary quantities corresponding to the unknown inputs and the commonknown inputs as the input values to all artificial neural networks, andthe quantities which decrease the differences to the least values areobtained as the estimated result by the unknown quantities correctingmeans. Then, the quantities corrected by the unknown quantitiescorrecting means are taken out by the information taking out means. Thetaken out quantities are the unknown quantities which are objected forestimation.

More particularly, by carrying out the predetermined learning in eachartificial neural network, a condition is obtained in which eachartificial neural network performs a processing which is equivalent to anumerical formulae regulating a relationship between plural inputs andan output. And, learning of the partial differential function is carriedout in each obtained artificial neural network based upon the differencebetween the cumulative addition value of the function value outputs andthe teacher pattern, and the difference between the partial differentialvalue corresponding to the function value output and the partialdifferential value (partial differential value teacher pattern) which ispreviously obtained, by supplying known inputs which should essentiallybe unknown quantities and the teacher pattern to the artificial neuralnetwork. Consequently, plural processings the number of which isdetermined in correspondence to kinds of the unknown quantities, whichprocessings are equivalent to numerical formulae (processings and thelike equivalent to plural numerical formulae which have differentconstants to one another, respectively) and processings equivalent topartial differential functions of the numerical formulae can be carriedout in corresponding artificial neural network. Thereafter, the outputis obtained by carrying out corresponding processing in each artificialneural network, then the estimated value corresponding to themeasurement value is obtained by cumulatively adding the outputs of allartificial neural networks. Then, the difference between the estimatedvalue and the measurement value is calculated. By correcting the unknowninputs of each artificial neural network based upon the calculateddifference, the arbitrary quantities corresponding to the unknown inputsare varied so as to decrease the difference. When the varying processingof the unknown inputs is repetitively carried out, and the differencebecomes sufficiently small, the corrected quantities are taken out asthe unknown quantities estimated result.

An apparatus for estimating unknown quantities according to anotherembodiment of the invention comprises;

plural artificial neural networks each of which has multiple layerarrangement and has finished learning of a predetermined function,

partial differential value calculating means for obtaining variationquantity of a function value output as a partial differential value byvarying a known quantity which should correspond to an unknown input ofeach artificial neural network by a very small amount,

cumulative addition means for cumulatively adding the function valueoutputs each of which is output from each artificial neural network,

first difference calculating means for calculating a difference betweena predetermined teacher pattern which was previously obtained and thecumulative addition value,

second difference calculating means for calculating a difference betweena partial differential value corresponding to the function value outputand a partial differential value which was previously obtained, forevery artificial neural network,

partial differential function learning means for carrying out learningof a partial differential function based upon the difference between thepartial differential values, calculated by the second differencecalculating means, and the difference between the teacher pattern andthe cumulative addition value, calculated by the first differencecalculating means,

input unknown quantities correcting means for correcting arbitraryquantities corresponding to unknown inputs and for obtaining thecorrected quantities which reduce the differences to the least values,as estimated results of unknown quantities, wherein all artificialneural networks are supplied arbitrary quantities corresponding to theunknown inputs and the common known inputs as input values so as toobtain the cumulative addition value of the function value outputs, eachartificial neural network correcting the arbitrary quantities so as todecrease the difference between the cumulative addition value and ateacher pattern based upon the difference between the cumulativeaddition value and the teacher pattern, and a partial differential valuecorresponding to the function value output of the correspondingartificial neural network, and

information taking out means for taking out the quantities which arecorrected by the unknown quantities correcting means.

As to the apparatus for estimating unknown quantities according to thisembodiment, by supplying the known inputs and the known teacher patternto each artificial neural network which has a multiple layer arrangementand has finished the learning of the predetermined function, thedifference between the cumulative addition value of the function valueoutputs and the teacher pattern is obtained by the first differencecalculating means, and the output variation value is obtained by thepartial differential value calculating means as the partial differentialvalue by varying the known quantities by a very small amount whichquantities should correspond to unknown input information. Then,arbitrary quantities corresponding to the unknown inputs are correctedby the unknown quantities correcting means so as to decrease thedifference between the cumulative addition value and the teacher patternbased upon the partial differential value corresponding to the functionvalue output of the corresponding artificial neural network and thedifference between the teacher pattern and the cumulative addition valueof the function value outputs which are obtained by supplying thearbitrary quantities corresponding to the unknown inputs and the commonknown inputs as the input values to all artificial neural networks, andthe quantities which decrease the differences to the least values areobtained as the estimated result by the unknown quantities correctingmeans. Then, the quantities corrected by the unknown quantitiescorrecting means are taken out by the information taking out means. Thetaken out quantities are the unknown quantities which are objected forestimation.

More particularly, by carrying out the predetermined learning in eachartificial neural network, a condition is obtained in which eachartificial neural network performs a processing which is equivalent to anumerical formulae regulating a relationship between plural inputs andone output. Consequently, plural processings the number of which isdetermined in correspondence to the kinds of the unknown quantities,which processings are equivalent to numerical formulae (processings andthe like equivalent to plural numerical formulae which have differentconstants to one another, respectively) and processings equivalent topartial differential functions of the numerical formulae can be carriedout in a corresponding artificial neural network. Thereafter, the outputis obtained by carrying out corresponding processing in each artificialneural network, then the estimated value corresponding to to themeasurement value is obtained by cumulatively adding the outputs of allartificial neural networks. Then, the difference between the estimatedvalue and the measurement value is calculated. By correcting the unknowninputs of each artificial neural network based upon the calculateddifference, the arbitrary quantities corresponding to the unknown inputsare varied so as to decrease the difference. When the varying processingof the unknown inputs is repetitively carried out, and the differencebecomes sufficiently small, the corrected quantities are taken out asunknown quantities estimated results.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an apparatus for estimatingunknown quantities according to an embodiment of the present invention;

FIG. 2 is a schematic diagram illustrating a portion corresponding toone hierarchical perceptron in detail;

FIG. 3 is a diagram illustrating an example of an estimated errorevaluating function; and

FIG. 4 is a schematic diagram illustrating a portion corresponding toone hierarchical perceptron of an apparatus for estimating unknownquantities according to another embodiment of the present invention indetail.

BEST MODES FOR EXECUTING THE INVENTION

Referring to the attached drawings, we explain the present invention indetail.

FIG. 1 is a block diagram illustrating an apparatus for estimatingunknown quantities according to an embodiment of the present invention,while FIG. 2 is a schematic diagram illustrating a portion correspondingto one hierarchical perceptron in detail.

The apparatus for estimating unknown quantities includes pluralhierarchical perceptrons 1i (i=1, 2, . . . , m), a sigma unit 2 forcumulatively adding function value outputs gij from the hierarchicalperceptrons 1i, a function value error operating section 3 for receivinga cumulative addition result Oj(t) output from the sigma unit 2 and ameasurement value Sj(t) as a teacher pattern, and for calculating adifference between the both, partial differential value calculatingsections 1ik (k=1, 2, . . . n) for calculating partial differentialvalues corresponding to the function value outputs gij from thehierarchical perceptrons 1i, partial differential value error operatingsections 3ik for receiving outputs from the partial differential valuecalculating sections 1ik of each hierarchical perceptron 1i and partialdifferential values (a partial differential value teacher pattern) whichare previously calculated by a numerical differential method and thelike, and for calculating a difference between the both, partialdifferential function learning sections 6i for carrying out learning ofpartial differential functions in the corresponding hierarchicalperceptron 1i based upon the difference calculated by the function valueerror operating section 3 and the differences calculated by the partialdifferential value error operating sections 3ik, correcting sections 12ifor correcting inputs at an input layer of the hierarchical perceptron1i based upon the difference between the cumulative addition resultOj(t) which is calculated by the function value error operating section3 and the measurement value Sj(t) as the teacher pattern, and thepartial differential values calculated by the partial differential valuecalculating sections 1ik, a control section 4 for selecting thecorrecting section 12i and the partial differential function learningsection 6i and for repetitively operating the selected correctingsection 12i and the partial differential function learning section 6i bya predetermined number of times (a number of times which decreases thedifference to be sufficiently small), and a collecting unit 5 foroutputting the inputs corresponding to unknown quantities of eachhierarchical perceptron 1i under a condition that the correctingprocessing is repeated by a predetermined number of times by thecorrecting sections 12i.

Each hierarchical perceptron 1i is a hierarchical perceptron in whichlearning of the function is sufficiently carried out by supplying aknown input pattern and a corresponding teacher pattern. Thehierarchical perceptron 1i performs processing corresponding to thenumerical formula which is determined based upon the input pattern andthe teacher pattern. The numerical formula may not actually be expressedby a numerical formula, or may be a numerical formula which requiresmuch labor for expressing with a numerical formula. Of course, thenumerical formula may be a numerical formula which is already expressedwith a numerical formula. A number m of the hierarchical perceptron 1iis determined in correspondence to a number of unknown quantities whichare objected for estimation. Further, the hierarchical perceptron 1i maybe controlled to operate in synchronism, respectively. The hierarchicalperceptron 1i also may be controlled to operate in asynchronism.

The partial differential function learning section 6i alternatelyrepeats learning (for example, back propagation learning) of thehierarchical perceptron 1i based upon the difference calculated by thefunction value error operating section 3 and learning of thehierarchical perceptron 1i based upon the difference calculated by thepartial differential value error operating sections 3ik. The partialdifferential function learning section 6i performs learning of thepartial differential function with slightly missing the learning resultof the function.

Further, the correcting section 12i includes correcting sections 12ikthe number of which is equal to the number of unknown quantities of thecorresponding artificial neural network 1i. The correcting section 12ikreceives the difference calculated by the function value error operatingsection 3 and the difference calculated by the partial differentialvalue operating section 1ik and corrects the corresponding unknownquantity.

Operation of the apparatus for estimating unknown quantities having theabove-mentioned arrangement is as follows.

Back propagation learning is performed in each artificial neural network1i by supplying the known inputs such as measurement conditions and theknown inputs corresponding to unknown quantities to the input layer ofeach artificial neural network 1i and by supplying the known measurementvalue determined based upon these known inputs to the artificial neuralnetwork 1i as the teacher pattern, so that weighting factors andthreshold values of each neuron device which constitutes the artificialneural network 1i are determined. When the back propagation learning issufficiently performed, processing corresponding to a numerical formulawhich is not actually expressed with a numerical formula is determinedin each artificial neural network 1i. That is, the output pattern gij ofthe artificial neural network 1i is expressed with the followingequation.

    gij=gi(t, ai1, ai2, . . . , aiL)                           (1)

In the equation, t represents time, and ai1, ai2, . . . , aiL representunknown quantities.

After the back propagation learning (learning of the numerical formula)has been finished in each artificial neural network 1i in theabove-mentioned manner, one of the known inputs corresponding to theunknown quantities among the input pattern is varied by a very smallamount so as to obtain an output pattern and to obtain correspondingpartial differential value by the partial differential value calculatingsection 1ik. Further, a partial differential value in a case that theknown input corresponding to the unknown quantity is varied by a verysmall amount, is previously calculated by the numerical differentialmethod and the like and is supplied as the teacher pattern. In thiscondition, the partial differential function learning section 6i isselected by the control section 4, and the selected partial differentialfunction learning section 6i is repetitively operated by a predeterminednumber of times (a number of times which decreases the difference to asufficiently small amount) so that the back propagation learning isperformed again. Thereby, the weighting factors and the threshold valuesof each neuron device which constitutes each artificial neural network1i are determined so as to perform the processing which is equivalent tothe operation of the equation (1) and an operation of a partialdifferential function of the equation (1).

Further, learning of each artificial neural network 1i may be performedindividually. When plural artificial neural networks 1i performprocessings equivalent to functions and partial differential functionswhich differ from one another only in their constants, the weightingfactors and threshold values obtained by any artificial neural networkas the learning result may be employed as they are in other artificialneural networks as their weighting factors and threshold values.Thereby, a required time for learning is extremely shortened. And, whenthe obtained weighting factors and threshold values are employed as theyare, the learning of the function and the learning of the partialdifferential function are not needed for the other artificial neuralnetworks, thereby a required time for learning is extremely shortened.

After the necessary learnings (the learning of the function and thelearning of the partial differential function) have been finished in theabove-mentioned manner, each unknown quantity is estimated with highaccuracy by selecting the correcting sections 12i instead of the partialdifferential function learning sections 6 and by operating thecollecting section 5 by the control section 4.

When time t and known information such as measurement conditions or thelike are supplied to each artificial neural network 1i, each artificialneural network 1i outputs the predetermined output pattern gij(t). Inthis case, the cumulative addition value Oj(t) output from the sigmaunit 2 is represented by the following equation. ##EQU1##

In the initial stage, though unknown quantities are arbitrarilydetermined, the obtained cumulative addition value Oj(t) is differentfrom the actual measurement value Sj(t). Therefore, the differencebetween the measurement value Sj(t) and the cumulative addition valueOj(t) is calculated by the function value error operating section 3 asan estimated error di(t), and partial differential values of thecumulative addition values Oj(t) are calculated based upon an equation(3) by the partial differential value calculating sections 3ik. ##EQU2##

And, information for the unknown quantities are estimated with highaccuracy by correcting the information for the unknown quantities otherthan the known inputs of each artificial neural network 1i based upon anequation (4) by the correcting sections 12i. Further, εk represents alearning gain (correcting gain) of the unknown quantity ai.

    aik=aik+εk{Sj(t)-Oj(t)} ∂{gi(t, ai1, ai2, ai3, . . . aiL)}/∂aik!                                  (4)

That is, when the correcting processings by the correcting sections 12iare repeated, the estimated errors di(t) decrease, and finally theestimated errors di(t) reach nearly 0. Therefore, an analyzing resultfor the unknown quantities are obtained by collecting and outputtinginformation for unknown quantities of each artificial neural network 1iat the timing by the collecting section 5.

Description is made in more detail in a point that estimation of unknownquantities is performed by repeating the processing in the equation (4).

When an estimated error evaluating function Ej(t) is defined by thefollowing equation, an equation (5) is obtained.

    Ej(t)=(1/2){Sj(t)-Oj(t)}.sup.2 ∂Ej(t)/∂Oj(t)=-{Sj(t)-Oj(t)}    (5)

When it is supposed that correcting of unknown quantities in eachphysical formula operating unit is performed based upon a maximum slopedecreasing method, estimation of unknowns for minimizing a value of theestimated error evaluating function can be performed based upon theequation (6). ##EQU3##

FIG. 3 is a diagram illustrating an example of an estimated errorevaluating function.

Estimating operation of unknowns which minimize a value of the estimatederror evaluating function is described, together with FIG. 3 and a table1 which indicates conditions of the estimated error evaluating function.In the chart 1, Δaik is a correction value according to the unknownquantity aik.

                  TABLE 1                                                         ______________________________________                                        aik                                                                                   ##STR1##                                                                                              ##STR2##                                      ______________________________________                                        Ej (t)                                                                                ##STR3##                                                                              local maximum                                                                           ##STR4##                                                                             local minimum                                                                          ##STR5##                            ∂Ej (t)                                                          ∂aik                                                                    positive 0        negative                                                                              0       positive                             Δaik                                                                           negative undefined                                                                              positive                                                                              0       negative                             ______________________________________                                    

As being mentioned in the foregoing, it is sufficient that theestimating of unknown quantities is performed to decrease the value ofthe estimated error evaluating function. Therefore, the correction valueΔaik is determined to be negative when a slope of the estimated errorevaluating function is positive, and is determined to be positive when aslope of the estimated error evaluating function is negative, by takinga sign of the slope of the estimated error evaluating function Ej(t). Itmay be thought that an unknown quantity aik corresponding to a localminimum is obtained when the above-mentioned estimation of unknownquantities is performed, because FIG. 3 includes a local minimum point.The estimation processing is not performed for only one unknownquantity, but is performed for all unknown quantities in a synchronousmanner, thereby the estimated error evaluating function itself is variedfollowing repetition of the estimation processing. As a result, unknownquantities which minimize the value of the estimated error evaluatingfunction are obtained finally. Thereafter, estimation of unknownquantities can be performed by collecting and outputting the finallyobtained unknown quantities by the collecting unit 5.

As is apparent from the foregoing description, when it is known that aconstant relationship between cause and effect exist between an inputpattern and an output pattern, and the relationship is not expressedwith a numerical formula, the apparatus for estimating unknownquantities according to the embodiment obtains a condition which isequivalent to a condition that the relationship is expressed with anumerical formula by constructing the relationship between cause andeffect which is not expressed by a numerical formula, in each artificialneural network 1i. Then, the apparatus for estimating unknown quantitiesperforms estimating processing based upon the obtained condition.Therefore, unknown quantities are estimated with high accuracy in acondition that only the known measurement condition and the like and themeasurement value are given, for example.

Further, when it is known that the relationship between cause and effectexisting between the input pattern and the output pattern can beexpressed with a formula, and that operating load becomes extremelygreat because the formula includes integration operations, convolutionoperation and the like, operation load is extremely decreased because acondition which is equivalent to a condition that the relationship isexpressed by a formula, is obtained, and estimating processing iscarried out based upon the obtained condition in each artificial neuralnetwork. Thereby, unknown quantities are estimated with high accuracyunder a conditions that only known measurement condition and the likeand measurement values are given, for example. Consequently, directivityof a sensor and the like are estimated with high accuracy using theapparatus for estimating unknown quantities according to the embodiment,for example.

Second Embodiment

FIG. 4 is a block diagram illustrating an apparatus for estimatingunknown quantities according to another embodiment of the presentinvention.

The embodiment differs from the above-mentioned embodiment in that inputvarying sections 15ik for varying only one input by a very small amount,output holding sections 16ik for holding outputs before and after thevarying of the one input, and difference calculating sections 17ik forcalculating a difference between the both outputs are provided insteadof the partial differential value calculating sections 1ik which areprovided in each artificial neural network in the above-mentionedembodiment.

In this embodiment, variation in output corresponding to the variationof the one input by a very small amount is calculated without performingpartial differential operation. Though the variation in output isequivalent to the partial differential value, function which is similarto that of the above-mentioned embodiment is performed. In thisembodiment, a required time for learning is shortened in comparison withthe above-mentioned embodiment, because learning of the partialdifferential function is not needed in this embodiment.

POSSIBILITY OF INDUSTRIAL UTILIZATION

As is apparent from the foregoing, methods and apparatus for estimatingunknown quantities according to the present invention, can calculateunknown quantities such as physical quantities and the like of aphysical source with high accuracy and with extremely short time period,in comparison with conventional analyzing apparatus which use asupercomputer and the like, based upon measured values of a physicalquantity, which values being obtained at plural points apart from thephysical source such as a magnetic field source and others, andobservation condition at the measurement timing, Further, the methodsand apparatus extremely shorten a required time for learning inartificial neural networks.

What is claimed is:
 1. A method for estimating unknown quantities,comprising:supplying arbitrary values corresponding to unknownquantities to each of a plurality of neural networks as a first group ofinputs, supplying known values to each of the neural networks as asecond group of inputs, supplying a predetermined measurement value tothe plurality of neural networks as a teacher pattern, each of theneural networks having multiple layers and a completed learning of apredetermined function, repeatedly correcting the arbitrary values foreach neural network, based upon partial differential valuescorresponding to the function value output of the neural network, so asto decrease a difference between the teacher pattern and a cumulativeaddition of the function value output for each neural network, andobtaining the corrected arbitrary quantities which reduce the differencebetween the teacher pattern and the cumulative addition of the functionvalue output for each neural network to the least value as estimatedresults of the unknown quantities.
 2. A method for estimating unknownquantities comprising:supplying known values as inputs to each of aplurality of neural networks to obtain a function value output for eachneural network and a plurality of partial differential valuescorresponding to the function value output for each neural network, eachof the plurality of neural networks having multiple layers and acompleted learning of a predetermined function; repeatedly obtaining afirst difference between a cumulative addition of the function valueoutput for each neural network and a previously obtained measurementvalue corresponding to the known inputs, andfor each of the plurality ofneural networks, obtaining a second difference between the partialdifferential values corresponding to the function value output for theneural network and previously obtained partial differential valuescorresponding to the known inputs, and learning a partial differentialfunction for the neural network based upon the obtained differences,until the obtained differences become sufficiently small; supplyingarbitrary values corresponding to unknown quantities to each of theneural networks as a first group of inputs; supplying known input valuesto each of the neural networks as a second group of inputs, supplying apredetermined measurement value corresponding with the known inputvalues to the neural networks as a teacher pattern, repeatedlycorrecting the arbitrary values for each neural network, based upon thepartial differential values for the neural network, so as to decrease athird difference between the teacher pattern and a cumulative additionof the function value output from each of the neural networks, andobtaining the corrected arbitrary values, which reduce the thirddifference between the teacher pattern and the cumulative addition ofthe function value from each of the neural networks to the least value,as estimated results of the unknown quantities.
 3. An apparatus forestimating unknown quantities, comprising:a plurality of neuralnetworks, each of the neural networks having multiple layers a completedlearning of a predetermined function and producing a function valueoutput in response to inputs, partial differential value calculatingmeans corresponding to each neural network, for calculating partialdifferential values corresponding to the function value output of theneural network, cumulative addition means for cumulatively adding thefunction value output of each neural network to produce a cumulativeaddition value, first difference calculating means for calculating afirst difference between a previously obtained teacher pattern and acumulative addition value produced by the cumulative addition means,unknown quantities correcting means corresponding to each neuralnetwork, for correcting arbitrary values corresponding to unknown inputsof the neural network, based upon partial differential values producedby the partial differential value calculating means, so as to reduce afirst difference produced by the first difference calculating means. 4.An apparatus for estimating unknown quantities as recited in claim 3,further including:second difference calculating means corresponding toeach neural network, for calculating a second difference between eachpartial differential value produced by the partial differential valuecalculating means for the neural network and a partial differentialvalue of the teacher pattern, and partial differential function learningmeans for carrying out learning of partial differential functions forthe partial differential value calculating means based upon a firstdifference produced by the first difference calculating means and asecond difference produced by the second difference calculating means.5. A method for estimating unknown quantities, comprising:supplyingarbitrary values corresponding to unknown quantities to each of aplurality of neural networks as a first group of inputs and supplyingknown values to each of the neural networks as a second group of inputs,supplying a predetermined measurement value to the plurality of neuralnetworks as a teacher pattern, each of the neural networks havingmultiple layers and a completed learning function, andrepeatedlyobtaining a first difference between a cumulative addition ofa first function value output for each neural network and apredetermined teacher pattern, for each neural network, varying only onearbitrary value for the neural network by a very small amount, such thatthe neural network produces a second function value output, obtaining asecond difference between the first function value output and the secondfunction value output, and correcting the arbitrary values, based uponthe second difference, so as to decrease the first difference, andobtaining the corrected arbitrary quantities which reduce the seconddifference for each neural network to a least value.
 6. An apparatus forestimating unknown quantities comprising:a plurality of neural networks,each of the neural networks having multiple layers and a completedlearning function and producing a first function value output inresponse to first inputs, cumulative addition means for cumulativelyadding a first function value output from each neural network to producea cumulative addition value, input varying means corresponding to eachof the neural networks, for repeatedly varying only one of the firstinputs to the neural network by a small amount, such that the neuralnetwork repeatedly produces a second function value output; firstdifference calculating means for calculating a first difference betweena previously obtained teacher pattern and a cumulative addition valueproduced by the cumulative addition means, second difference calculatingmeans corresponding to each of the neural networks, for calculating adifference between a first function value output of the neural networkand a second function value output of the neural network; and correctionmeans corresponding to each neural network for correcting arbitraryvalues corresponding to unknown inputs of the neural network, based upona second difference produced by the second difference calculating means,so as to reduce a first difference produced by the first differencecalculating means.
 7. The apparatus for estimating unknown quantities asrecited in claim 6, further including output holding means correspondingto each of the neural networks, for holding both a first function valueoutput of the neural network and a second function value output of theneural network.